1. Use the DISCUSSION tab above to find your discussion questions and/or tasks.
2. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Binary Choice activities will be the member whose first name comes earliest in alphabetical order.):
Binary Choice Item # 1 - flaw(s)? When reading this sentence we first noticed that is refers to Professional Development and NCLB (No Child Left Behind). These are two different topics which could confuse the students when trying to answer true/false. We also noticed their use of 'they'. 'They' could be referring to district administrators or teachers. Using 'they' could confuse the students and change the meaning of the question. Also the aim to see what the students do and do not know, not to try and trick them.
Evidence: Commandment 2: This question goes against Commandment 2 in that the "sentence in which pronouns are used can be unclear as to whom the pronoun refers." It also goes against "avoid items that include more than one concept" in this sentence that would be Professional Development and NCLB.
Binary Choice Item # 2 - flaw(s)?
When reading this sentence we found two mistakes. One, was the word choice for this sentence. The words are very difficult and because of that the students would be at a disadvantage with answering this question. Two, was the length and complexity of this sentence. With the words being as difficult as they are it does not help that the sentence is long and complex. Since the students would most likely be unable to pronounce majority of the words we doubt they would read the entire sentence, they would get frustrated.
Evidence: This question first violates the 4th Commandment "avoid complicated sentence construction - use very simple sentences." Secondly, it violates the 5th Commandment "use vocabulary that is suitable for the students who will be taking your test."
Binary Choice Item # 3 - flaw(s)?
When reading this sentence we came across 3 flaws. First, the sentence goes against Commandment 3 by giving unintended clues; the first part of the statement seems to give away the answer. Secondly, the length of the sentence could make it hard for the students to concentrate on answering the question, especially if they have to go back and reread the question to get full understanding. Thirdly, the use of 'impossible' could confuse and/or give away the answer to the question.
Evidence:
This question goes against Commandment 3 "Thou shall not provide students with unintended clues regarding appropriate responses." Also this question goes against Maintaining a similar length for both categories of items "you don't want to give away clues with the item length." Finally, 'impossible' would be an absolute qualifier and should not be in the sentence because you're trying to assess the students not trick them.
Binary Choice Item # 4 - flaw(s)?
When reading this sentence we saw that 'never' is used in the statement. By having 'never' in the sentence could change the students perception of the statement and could also confuse them. There were also a few words in the sentence that a student may not know which could hinder their understanding of the statements meaning.
Evidence:
This statement goes against Commandment 3 by using an absolute qualifier 'never' and also Commandment 5 by using language that could be above the students grade level and understanding. This use of negatives in this sentences makes the actual question unclear to the students.
3. POST YOUR GROUP PRODUCT HERE - Multiple Binary-Choice assessment item (the group member whose birthday is closest to Christmas will be the leader for this Multiple Binary-Choice activity :
Read the statement below and answer each statement with either true or false:
Jan, Bob, Lucy, and Pete are in line to see a movie. Lucy is not in the front of the line. Bob and Jan are on either side of Lucy. Bob is directly behind Pete.
True or False:
1. Jan is at the end of the line. 2. Lucy is second in line. 3. Pete is at the front of the line. 4. Bob is third in line.
Answers: 1) True 2) False 3) True 4) False
4. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Multiple Choice activities will be the member whose last name comes earliest in alphabetical order.):
Multiple-Choice Item # 1 - flaw(s)?
When reading this multiple-choice question we noticed the use of some advanced vocabulary; for example, 'succinct' and 'palatable'. Also the stem and alternatives are too long. When writing a multiple-choice question it is preferred to have a length stem and terse alternatives; not a skimpy stem and long alternatives.
Evidence:
This question goes against Commandment 5 "Thou shall not use vocabulary that is more advanced than your audience. It also goes against the Writing Guidelines for multiple-choice questions. The stem and alternatives are too long and poorly written.
Multiple-Choice Item # 2 - flaw(s)?
When reading this statement we thought it was a little wordy but otherwise fine. The answer choices are what we found major flaws in. All of the answers seem to be close to the same meaning which could really throw students off; for example, two of the answers have the word 'standard' in them. The student could know the answer has to do with standards and could be confused with two answers containing that word. Also the statement ends in 'described as an:' A student who knows their grammar will know the answer automatically.
Evidence:
The statement goes against Commandment 3 "Thou shall not provide students with unintended clue regarding appropriate responses."
Multiple-Choice Item # 3 - flaw(s)?
In this question be noticed the use of 'all of the above'. The use of 'all of the above' should be avoided when writing multiple-choice questions because it can really confuse the students.
Evidence:
This question goes against the Writing Guidelines for multiple-choice questions. The powerpoint states "that is the student knows two the the answer are correct, he/she is likely to assume the other choices are correct as well."
Multiple-Choice Item # 4 - flaw(s)?
When reading this question, the question itself it fine but the responses are where we found the flaws. Choices A, B, and C are all simply worded and at a decent length. However, choice D clearly stands out because of the length. It would be apparent to a student that D would be the correct answer based on its length.
Evidence:
This question goes against the Writing Guidelines for multiple-choice questions in that "item writers frequently fail to realize that the length of a multiple-choice item can give an unintended clue to the correct answer." Students are likely to realize that D is the correct answer because the teacher put a lot of detail in choice D compared to the other three choices. This could be a disadvantage to the teacher because he/she will not know if the student really knows the material or not because of the unintended clue.
5. POST YOUR GROUP CONSENSUS HERE (The group leader for this Matching Item activity will be the member whose birthday is closest to Valentine's Day.):
“Find the flaws . . . use evidence from the ‘5 Commandments and/or the Item Writing Guidelines to support your answers.” On the line to the left of each measurement concept listed in Column A, please write the letter of the most relevant measurement procedure in Column B.
Column A
Column B
1. Bias-Detection
A. Determining DIF indices
2. Item Difficulty
B. Computing K-R formula
3. Reliability
C. Evaluating curricular alignment
4. Score Interpretation
D. Deriving percentiles
5. Validity
E. Calculating p// - values
Matching Item # 1 - flaw(s)?
When looking at this Matching Item example we found a few mistakes. First, the directions are poorly written. We believe they should say "Please match each letter in column B with the appropriate answer in column A Each response can only be used one time." Secondly, the number of premises and responses are the same. Thirdly, we do not think the responses are in any significant order.
Evidence:
This matching item goes against the Writing Guidelines for Matching Items. Directions should always be clearly stated because if not the students make be unclear to what is being asked of them. The powerpoint states that responses should be put in some type of significant order, alphabetically, for example. And the powerpoint also states that there should be a few more responses than premises to challenge the student. A student who knows 80% of the answers could easily eliminate others and figure out the answers. That would not be an accurate show of what the student knows.
1. Use the DISCUSSION tab above to find your discussion questions and/or tasks.
2. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Binary Choice activities will be the member whose first name comes earliest in alphabetical order.):
Binary Choice Item # 1 - flaw(s)?
When reading this sentence we first noticed that is refers to Professional Development and NCLB (No Child Left Behind). These are two different topics which could confuse the students when trying to answer true/false. We also noticed their use of 'they'. 'They' could be referring to district administrators or teachers. Using 'they' could confuse the students and change the meaning of the question. Also the aim to see what the students do and do not know, not to try and trick them.
Evidence: Commandment 2:
This question goes against Commandment 2 in that the "sentence in which pronouns are used can be unclear as to whom the pronoun refers." It also goes against "avoid items that include more than one concept" in this sentence that would be Professional Development and NCLB.
Binary Choice Item # 2 - flaw(s)?
When reading this sentence we found two mistakes. One, was the word choice for this sentence. The words are very difficult and because of that the students would be at a disadvantage with answering this question. Two, was the length and complexity of this sentence. With the words being as difficult as they are it does not help that the sentence is long and complex. Since the students would most likely be unable to pronounce majority of the words we doubt they would read the entire sentence, they would get frustrated.
Evidence:
This question first violates the 4th Commandment "avoid complicated sentence construction - use very simple sentences." Secondly, it violates the 5th Commandment "use vocabulary that is suitable for the students who will be taking your test."
Binary Choice Item # 3 - flaw(s)?
When reading this sentence we came across 3 flaws. First, the sentence goes against Commandment 3 by giving unintended clues; the first part of the statement seems to give away the answer. Secondly, the length of the sentence could make it hard for the students to concentrate on answering the question, especially if they have to go back and reread the question to get full understanding. Thirdly, the use of 'impossible' could confuse and/or give away the answer to the question.
Evidence:
This question goes against Commandment 3 "Thou shall not provide students with unintended clues regarding appropriate responses." Also this question goes against Maintaining a similar length for both categories of items "you don't want to give away clues with the item length." Finally, 'impossible' would be an absolute qualifier and should not be in the sentence because you're trying to assess the students not trick them.
Binary Choice Item # 4 - flaw(s)?
When reading this sentence we saw that 'never' is used in the statement. By having 'never' in the sentence could change the students perception of the statement and could also confuse them. There were also a few words in the sentence that a student may not know which could hinder their understanding of the statements meaning.
Evidence:
This statement goes against Commandment 3 by using an absolute qualifier 'never' and also Commandment 5 by using language that could be above the students grade level and understanding. This use of negatives in this sentences makes the actual question unclear to the students.
3. POST YOUR GROUP PRODUCT HERE - Multiple Binary-Choice assessment item (the group member whose birthday is closest to Christmas will be the leader for this Multiple Binary-Choice activity :
Read the statement below and answer each statement with either true or false:
Jan, Bob, Lucy, and Pete are in line to see a movie. Lucy is not in the front of the line. Bob and Jan are on either side of Lucy. Bob is directly behind Pete.
True or False:
1. Jan is at the end of the line.
2. Lucy is second in line.
3. Pete is at the front of the line.
4. Bob is third in line.
Answers: 1) True 2) False 3) True 4) False
4. POST YOUR GROUP CONSENSUS HERE (The group leader for these 4 Multiple Choice activities will be the member whose last name comes earliest in alphabetical order.):
Multiple-Choice Item # 1 - flaw(s)?
When reading this multiple-choice question we noticed the use of some advanced vocabulary; for example, 'succinct' and 'palatable'. Also the stem and alternatives are too long. When writing a multiple-choice question it is preferred to have a length stem and terse alternatives; not a skimpy stem and long alternatives.
Evidence:
This question goes against Commandment 5 "Thou shall not use vocabulary that is more advanced than your audience. It also goes against the Writing Guidelines for multiple-choice questions. The stem and alternatives are too long and poorly written.
Multiple-Choice Item # 2 - flaw(s)?
When reading this statement we thought it was a little wordy but otherwise fine. The answer choices are what we found major flaws in. All of the answers seem to be close to the same meaning which could really throw students off; for example, two of the answers have the word 'standard' in them. The student could know the answer has to do with standards and could be confused with two answers containing that word. Also the statement ends in 'described as an:' A student who knows their grammar will know the answer automatically.
Evidence:
The statement goes against Commandment 3 "Thou shall not provide students with unintended clue regarding appropriate responses."
Multiple-Choice Item # 3 - flaw(s)?
In this question be noticed the use of 'all of the above'. The use of 'all of the above' should be avoided when writing multiple-choice questions because it can really confuse the students.
Evidence:
This question goes against the Writing Guidelines for multiple-choice questions. The powerpoint states "that is the student knows two the the answer are correct, he/she is likely to assume the other choices are correct as well."
Multiple-Choice Item # 4 - flaw(s)?
When reading this question, the question itself it fine but the responses are where we found the flaws. Choices A, B, and C are all simply worded and at a decent length. However, choice D clearly stands out because of the length. It would be apparent to a student that D would be the correct answer based on its length.
Evidence:
This question goes against the Writing Guidelines for multiple-choice questions in that "item writers frequently fail to realize that the length of a multiple-choice item can give an unintended clue to the correct answer." Students are likely to realize that D is the correct answer because the teacher put a lot of detail in choice D compared to the other three choices. This could be a disadvantage to the teacher because he/she will not know if the student really knows the material or not because of the unintended clue.
5. POST YOUR GROUP CONSENSUS HERE (The group leader for this Matching Item activity will be the member whose birthday is closest to Valentine's Day.):
“Find the flaws . . . use evidence from the ‘5 Commandments and/or the Item Writing Guidelines to support your answers.”
On the line to the left of each measurement concept listed in Column A, please write the letter of the most relevant measurement procedure in Column B.
Matching Item # 1 - flaw(s)?
When looking at this Matching Item example we found a few mistakes. First, the directions are poorly written. We believe they should say "Please match each letter in column B with the appropriate answer in column A Each response can only be used one time." Secondly, the number of premises and responses are the same. Thirdly, we do not think the responses are in any significant order.
Evidence:
This matching item goes against the Writing Guidelines for Matching Items. Directions should always be clearly stated because if not the students make be unclear to what is being asked of them. The powerpoint states that responses should be put in some type of significant order, alphabetically, for example. And the powerpoint also states that there should be a few more responses than premises to challenge the student. A student who knows 80% of the answers could easily eliminate others and figure out the answers. That would not be an accurate show of what the student knows.